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OPTIMUM  SAMPLING  TIMES  FOR  SPECTRAL  ESTIMATION 

by 

Lonnie  C.  Ludeman 


ABSTRACT 

The  problem  of  optimum  sampling  strategies  for  spectral  estimation 
of  Fourier-type  signals  in  the  case  of  finite  discrete-time  observations 
was  investigated.  In  particular,  it  was  shown  that  minimum  variance 
unbiased  estimates  of  amplitudes  of  sine  and  cosine  terms  of  Fourier 
signals  embedded  in  additive  zero-mean  white  noise  can  be  determined  by 
sampling  at  the  generalized  Chebyshev  times.  The  solution  obtained, 
by  putting  the  problem  in  an  optimum  linear  regression  framework,  is 
that  the  generalized  Chebyshev  times  are  the  zeros  of  the  derivative 
of  the  highest  frequency  cosine  wave.  If  the  number  of  samples  exceeds 
the  intrinsic  dimensionality,  repeated  independent  sampling  at  those 
points  not  only  provides  the  best  approximation  to  the  Fourier  signal  in 
a  minimum  variance  sense  but  also  the  linear  minimum  variance  unbiased 
estimate  of  the  coefficients. 
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I. 


INTRODUCTION 


In  obtaining  spectral  information  from  noisy  discrete-time 
measurements  of  Fourier  signals,  a  basic  question  is  whether  performanc 
can  be  improved  by  carefully  selecting  the  sampling  times.  Preliminary 
results  shown  by  the  author  [8,9]  in  pursuit  of  the  answer  to  this  ques 
tion  helped  formulate  the  main  objectives  of  this  research  effort.  The 
objectives  were: 

(1)  Find  the  optimum  sampling  strategy  for  the  case  of 
minimum  variance  unbiased  estimation  of  (a)  the 
amplitudes  of  several  sine  and  cosine  terms  of  har¬ 
monically  related  frequencies  in  white  noise; 

(b)  the  amplitudes  of  Fourier-type  nonharmonically 
related  signals  in  additive  white  noise;  and  (c) 
spectral  content  of  a  low  pass  process  in  additive 
white  noise. 

(2)  Repeat  (1)  for  special  types  of  nonwhite  additive  noise. 

(3)  Find  the  improvement  obtained  with  nonuniform  over- 
sampling  for  the  cases  given  in  (1)  a,  b,  and  (2)  a,  b. 

(4)  Show  if  possible  under  what  conditions  the  uniform 
sampling  value  estimates,  determined  from  nonuniform 
sample  times,  provide  a  sufficient  statistic  for  the 
spectral  estimation  problem. 

During  the  grant  period  progress  was  made  in  areas  specified  in 
Objectives  1,  2  and  3,  while  the  sufficient  statistic  problem  given 
in  4  has  not  been  resolved.  In  the  sections  to  follow,  optimum 
sampling  strategies  will  be  shown  for  both  Fourier  signals  and 
Fourier-type  signals  in  additive  white  noise  ((1)  a,  b)  from  two 
different  approaches. 

The  first  approach,  Che  main  body  of  the  report,  places  the 
problem  in  a  linear  regression  framework  and  uses  the  elegant  and 
brilliant  work  of  Kiefer  and  Wolfowitz  [2]  to  arrive  at  the  solution 
that  the  optimum  sample  times  are  specified  by  a  generalized  Chebyshev 
set . 
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The  second  approach — more  of  a  brute  force  method — is  that  of 
minimizing  the  trace  of  the  error  covariance  matrix  as  presented  in 
Appendix  A.  Optimum  sampling  points  were  obtained  by  applying 
standard  search  and  steepest  descent  optimization  algorithms  to 
the  multidimensional  problem.  This  approach  was  complicated  by  the 
dimensionality,  the  intrinsic  dimensionality,  and  the  preponderance 
of  local  rather  than  global  minimums. 

The  nonwhite  additive  noise  problem  was  approached  again  both 
by  computer  simulation  (Appendix  A)  and  from  the  standpoint  of  changing 
the  colored  noise  to  white  noise  via  a  bleaching  filter  and  searching 
for  the  Chebvshev  points  of  the  bleached  basis.  Computer  results  showed 
equal  spacing  for  the  three  sample  case  of  a  signal  composed  of  a 
sum  of  a  sine  and  cosine  wave  for  colored  noise  with  exponential  auto¬ 
correlation  function. 

Progress  on  Objective  (3)  has  been  minimal,  but  because  of  results 
of  Kiefer  and  Wolfowitz  [2]  it  is  believed  that  for  large  numbers  of 
samples,  much  more  than  the  intrinsic  dimensionality  of  the  problem, 
uniform  sampling  strategy  requires  roughly  8/ir  times  the  number  of 
samples  in  the  optimum  strategy  to  yield  identical  performance.  The 
optimal  sampling  strategy  may  in  some  sense  not  be  physically  meaningful 
in  that  it  requires  taking  more  than  one  sample  at  several  points  or 
times:  however,  because  of  periodicity  of  the  assumed  signals  this 
could  mean  simply  using  samples  in  a  later  period. 
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II.  PROBLEM  FORMULATION 

The  common  problem  of  spectral  estimation  involves  the  estimation 

of  the  frequency  content  of  a  given  signal.  Normally  the  signal  is  not 

available  over  all  time  and  has  been  contaminated  with  additive  noise. 

If  we  assume  further  that  the  processing  is  to  be  done  digitally,  we 

have  available  only  measured  values  at  some  finite  set  of  times.  The 

problem  we  wish  to  address  is  that  of  optimum  sampling  strategies  for 

til 

spectral  estimation  of  Fourier-type  signals.  The  K  order  Fourier 
signal  (trigonometric  polynomial)  is  given  by 

xK(t)  =  f  + 

The  A^ , A^ , . . , and  B^.B^, . . . .B^  represent  unknown  nonrandom  Fourier 
coefficients.  It  is  further  assumed  that  we  have  available  z(t)  over  an 
interval  -1  to  1,  given  by 

z(t)  =  xR(t)  +  N(t)  -1  £  t  <_  1  (2) 

where  N(t)  is  a  random  noise  process  characterized  by  its  autocorrelation 
function.  Let  be  an  arbitrary  set  of  M  time  values  given  by 

M  =  *tl,t2,‘  "  ,t:M} 

Observing  z(t)  at  the  times  of  T  where  M  is  greater  than  or  equal  to 

M 

2K  +  1  gives  the  set  of  measurements 

z(t.)  =  xK(t.)  +  N(ti)  i  =  1,2 . M>  2K+1  (3) 

We  would  like  to  select  in  such  a  way  that  optimal  estimates  of 
Aq,A^, . . . .A^,  B^,B2»...,B^  or  XK(0  are  obtained  where  optimality  is 
with  respect  to  a  certain  performance  criterion  and  a  given  M.  Two 
prominent  criteria  are  1)  minimax  variant  which  is  to  minimize  the 
maximum  variance  of  the  estimate,  x^(t)  over  all  tef-1,1]  using  Lagrangian 
interpolation,  and  2)  minimum  trace  of  the  error  covariance  matrix,  that  is, 
minimizing  the  sum  of  the  variances  of  the  estimates  of  the  coefficients. 

The  first  criterion  will  be  used  in  the  body  of  this  report,  while  the 
results  of  the  second  are  given  in  the  appendix  for  completeness. 


E 

k=l 


A^coskirt 


B,  sinkirt 
k 


(1) 
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III.  OPTIMUM  POLYNOMIAL  REGRESSION 

The  optimum  spacing  problem  for  finite  polynomial  regression  has 
been  investigated  by  De  La  Garza  [A, 5],  Hoel  and  Levine  [6],  and  Kiefer 
and  Wolfowitz  [3].  The  framework  for  their  development  is  as  follows: 

Let  P(x)  be  a  polynomial  given  by 

P(x)  =  a,  +  a„x  +...+  a  ,,  xm  m  >  1  (4) 

I  2  m+1  — 

and  let  y(x^)  represent  observed  variates  given  by 

y(Xi)  =  P(xi)  +  6t  i  =  1,2, .. . ,N,  N  >  (m+1)  (5) 

where  6.  are  uncorrelated  random  errors  with  known  variances  and  zero  means. 

l 

It  was  shown  that  given  a  selection  of  N  >  (m+1)  distinct  points  x^, 
with  minimum  x^  =  -1,  maximum  x^  =  1,  it  is  always  possible  to  respace 
at  k+1  distinct  location  with  repeated  values  of  x^  to  yield  identical 
performance.  Therefore,  for  optimum  selection  you  need  only  find  those 
(m+1)  points  and  the  number  of  times  each  was  repeated.  The  points  of 
observation  and  the  number  of  times  n^  for  minimax  variance  as  given  by 
Kiefer  [2]  are 


xo  =  -1. 

.IT 

X,  =  -COST”, 

J  m’ 

j  -  1,  2, 

...»  m-1. 

x  =1,  with 
m 

(6) 

Hcn 

II 

o 

c 

N  .  . 

n.  =  -,  1  =  1, 
l  m 

•  •  •  9  III- 1  f 

II 

a  I55 

(7) 

This  result  will  now  be  applied  to  the  special  case  where  x  (t)  contains 

K 

only  harmonic  cosine  waves. 

Example  1.  Let  x  (t)  represent  a  series  of  harmonic  cosine  waves 

K.C 

given  by 

m 

vo-'.-Es  cosknt  (8) 

k=l 

It  is  well  known  that  each  of  the  cosine  waves  coskut  can  be  written  in 
terms  of  powers  of  cosirt,  the  fundamental,  by  repeated  use  of  the 
recursive  relation 

coskirt  =  2cos(k-l)tTt  cosTTt  -  cos(k-2)irt 

That  is,  x„  (t)  can  be  written  as  a  KC^  order  polynomial  in  coswt  given  by 
Kc 
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(9) 


x  (t)  =  a  +  a  cosTrt  +  a  (cosut)^  +  ...  +  a  (cosTTt)m 

ivC  O  1  z  K 

By  letting  cos  t  =  x,  we  have  the  -1  £  x  <_  1  and  the  polynomial  condition 

necessary  to  find  the  optimal  x  and  thus  the  corresponding  optimal  t 
from  the  polynomial  regression  results  given  by  equation  (6) .  This 
yields 

cosirt  =  -1  cosmt  =  -cosj—  j  =  1,  ...»  m-1  cosirt  =  1  (10) 

m 

Therefore  we  see  the  solution  for  the  t^,  after  reordering,  is  as  follows: 

t.  =  j  j  =  0,1,  ....  m  on  [0,1]  (11) 

Computer  results  using  the  formulation  given  in  Appendix  A  for 
the  case  of  a  cosine  wave  and  its  first  harmonic  showed  that  the  optimum 
points  on  the  interval  [0,1)  were  t  =  0.0  and  t  =  0.5  with  these  same  times 
repeated  when  the  number  of  points  taken  was  greater  than  2.  If,  however, 
harmonics  were  skipped — for  example,  using  only  the  first  and  third — these 
points  were  no  longer  optimum.  If  harmonics  are  skipped  or  not  present, 
the  intrinsic  dimensionality  of  the  problem  has  been  reduced  and  the 
resulting  cosine  functions  used  for  the  basis  in  the  generalized  regression 
are  no  longer  Chebyshev. 

Computer  results  showed  also  that  for  two  cosine  waves  not  har¬ 
monically  related,  the  optimum  times  were  eauallv  spaced  if  the  waves 
were  Chebyshev. 
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IV.  GENERALIZED  OPTIMUM  REGRESSION 


In  Kiefer  and  Wolfowitz  [2],  the  optimum  selection  of  regressor 
variables  was  extended  to  cover  other  nonpolynomial  regression  problems 
involving  Chebyshev  systems.  In  this  section  we  will  summarize  the  impor¬ 
tant  results  of  their  extension  and  later  apply  them  to  the  problems  of 
spectral  estimation. 

The  regression  problem  considered  assumes  the  observed 
function  z(t)  is  given  by 


z(t)  = 


■  E  v. 


(t)  +  N(t) 


-1  <  t  <  1 


(12) 


i=0 


where  the  f^(t)  are  continuous  and  linearly  independent  real-valued  functions 
on  [-1,1],  We  would  like  to  select  t^,  i=0,l, . . . ,n^.m+l  such  that  the 
maximum  variance  on  z(t)  over  T  is  minimized  where  T={t:-l<t<L). 

The  functions  fQ(t),  f^(t),...,  f^(t)  are  called  a  Chebyshev  system 


on  a  set  U  if  every  linear  combination 


m 

V  cifi(t)’ 

k=0 


with  not  all  of  the  real 


constants  c^  zero,  has  k  or  fewer  zeros  on  U.  Several  other  equivalent 
specifications  are  given  by  K  and  W  [ 3  ] .  We  shall  denote  B(c*)  to  be  the 


set  of  nri-1  points  where  f  (t) 

m 


m-1 

-  E 

k=0 


c*fk(t) |  attains  its  maximum  on  [-1,1] 


,* 

'm 

71-1 


and  c*  =  (c*,  c*,  ...,  c*  ,)  as  the  constants  c  =  (c  ,c, ,...,c  ,)  such 

o  1  m-1  o  1  m-1 


that  max  |f  (t)  - 
-l<t<l  m 


Sv. 

j=0 


(t) |  is  minimized. 


B(c*)  consists  of  the  m+1  points  given  by 

t*  =1,  t*  <  t*  <...<  t*  ,,  t*  *  1. 
o  m-!  m 

Kiefer  and  Wolfowitz  have  shown  for  the  given  assumptions  that 

the  c*  must  satisfy 


J1  m—  1 

[fm(t)  -  Y*  cjf  j  (t)  1  fi<t)C*(dx)  -  0  0  <  i  < 

-1  J=0 


m 


(13) 
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For  the  case  where  the  probability  measure  £  is  uniform,  we  have 


J1  m-1 

[fm(t)  -  V  c*f J (t) ]  f.(t)dt  =0  o  <  i 
-1  j^O 


l  <  m 


(14) 


If  we  further  assume  the  {f^(t),  j=0,l,...,m}  to  be  orthogonal  on  [-1,1], 


l. e. . 


r 

J  f^(t)f^(t)dt  =  j,k  =  0,1,...,  m 

^  where  w.  >0 
3 


then  the  set  of  equations  in  ( 14  )  reduce  to 

c*w.  =0  0  <  i  <  m 

ii  — 

Since  w.  are  nonzero  it  implies  that  c*  =  0  for  i=0,l, . . . ,m-l. 

3  JL  3 

With  this  result  B(c  )  is  seen  to  be  from  [3]  to  be  the  place  where 
|  f m ( t ) |  attains  its  maximum  values  on  (-1,1). 

Application  to  Spectral  Estimation  White  Noise 


(15) 


The  above  result  is  now  applied  to  the  case  where  f^(t)  are  the 
Fourier  Basis  functions  on  -1  <  t  <  1  given  by 

f  (t)  =  1 
o 

f^(t)  =  simrt 

f2(t)  =  COSTTt 

f^(t)  -  sin2irt 

f,  (t)  =  cos2Ttt 

f  *  (t)  =  sin(?)iTt 
m— i  i 

f  (t)  =  cos(y)irt 
m  z 

We  wish  to  select  the  times  t  such  that  the  maximum  variance  is 
minimized  throughout  -1  to  1.  The  result  states  that  the  optimum  times 
will  be  the  points  at  which  |cos^rrt|  reaches  maximum,  that  is,  where 
t  equals  -1  and  1  and  sin^irt  equals  zero-  Therefore  the  times  are  given  by 
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jir  "  2j /m 

■jut  =  ^  0  which  implies  t  =<  0 

-jir  -2j/m 


These  points  are  illustrated  in  Figure  1  below. 


J  1 » » • • j  m/ 2 

j  =  0,1, ,  m/2 


— » - -» - « - • - • - •-  - -  ♦ 

-1  -(m-2)/m  -4/m  -2/m  0  2 /m  4/m 


Ll  I 

m-1 


n 

m 


Optimum 

number 


u  ,  u 

m-1  m 

- ♦ - •- 

(m-2)/m  1 


t 


Optimum 

times 

Chebvshev 

points 


Figure  1.  Optimum  Sample  Times  for  Estimation  of  Fourier 

Signals:  m+1  Chebyshev  Points  with  N  Total  Samples. 


For  N  samples,  where  N  is  greater  than  m+1, 

points  will  have  weights  £  ,  £,  ,  ...»  £  given  by 

o  I  m 


?o  2m’ 


*i  -  s- 


l  —  1,2,...,  m- 1 , 


the  above  Chebyshev 


2m 


If  N  is  the  total  number  of  points,  then  ,  the  number  of  samples  to 

be  taken  at  t.,  is  N*£..  Furthermore,  if  we  let  N  =  2m*r  where  r  is  an 
J  J 

integer,  we  have 


n  =  r,  n.  =  2r,  j  =  l,2,...,m-l,  n  =  r 
o  l  m 

The  variance  at  each  point  and  thus  the  minimum  maximum  variance 

2  2 

becomes  a  / n.  where  a  is  variance  of  each  observation  and  n.  is  the 
J  J 

number  of  measurements  taken  at  each  point  in  the  interior  of  -1  and  1. 

When  N  is  a  multiple  of  2m  as  above,  we  get 

2  2 

.  .  .  mo  o 

minimax  variance  =  — —  =  — — 

N  2r 


Kiefer  and  Wolfowitz[2]  defined  a  sampling  efficiency  and  showed 
for  polynomial  regression  that  the  evenly  spaced  samples  strategy 
requires  roughly  8/v  or  about  5/2  times  that  of  Chebyshev  points 
strategy  for  large  N. 
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Application  to  Spectral  Estimation:  Colored  ?:oise 

If  the  N(t)  given  in  equation  (12)  is  not  white  but  its  autocorrelation 
function  is  known,  one  approach  to  finding  the  optimum  sampling  times  is  to 
change  the  problem  to  white  noise  by  "bleaching."  We  assume  that  N(t) 
can  be  obtained  by  passing  a  white  noise  process  w(t)  through  a  time 
invariant  linear  filter  with  system  function  H(s) .  The  whitening  filter 
would  be  the  reciprocal  of  H(s)  and  would  operate  on  z(t)  to  give  a  new 
measurement  process  zb(t)  shown  in  Figure  2. 


Figure  2.  Conversion  to  White  Noise  Problem  by  Bleaching. 

The  zb(t)  is  composed  of  w(t),  a  white  noise  process,  and  a  bleached 

version  xb(t)  of  x  (t).  If  we  allow  the  filtering  to  reach  a  steady 
K  K. 

state,  zb  (t)  can  be  written  as 
ss 


zbc(t)  =  *H  (0)  + 
ss  2  b 


K 

XjVjkWo 


)  I  cos  kw^t  +  tan 


1  HbI(jkWo)  \ 

HKD(jkW  J  / 


bR  o' 


x  HbI (ikWo) 

b(jkwo)  |sin(kwot  +  tan  +  w(t) 


+  w(t)  where  w  =  it 

o 


We  now  have  the  same  framework  as  before  for  optimum  linear 
regression.  It  is  easy  to  show  that  the  translated  sine  and  cosine 
functions  are  linearly  independent  and  Chebyshev,  thus  specifying  the 
optimum  sampling  times  for  the  steady  state  bleached  z(t).  The 
performance,  however,  of  the  estimates  will  be  modified  by  the  fact 
that  the  bleaching  process  changes  the  variance  of  the  N(t)  by 
| l/H ( j 0) !  .  Computer  results  for  the  case  of  colored  noise  with  auto¬ 
correlation  function  R.,(x)  =  e  aT  and  a  first  order  trigonometric 
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polynomial  showed  that  equally  spaced  samples  were  optimum.  Results  for 
higher  order  trigonometric  polynomials  and  over  sampling  were  not 
obtained. 

V.  CONCLUSION 

It  has  been  shown  that  optimum  sampling  strategies  exist  for 

estimating  Fourier  signals  using  a  finite  number  of  discrete  time 

observations.  By  framing  the  problem  as  a  generalized  regression 

with  a  Chebyshev  basis,  it  has  been  shown  that  the  generalized  Chebyshev 

times  are  the  zeros  of  the  highest  sine  wave  considered.  Therefore,  the 

samples  are  equally  spaced  with  the  number  of  unique  samples  being 

determined  by  the  dimensionality  of  the  trigonometric  polynomials. 

For  the  case  of  a  Fourier  series  of  order  m,  with  no  missing  harmonics, 

the  sample  points  on  -1  to  1  are  +  j  =  0,  ...,  m/2.  If  harmonics 

m 

were  missing,  the  dimensionality  of  the  problem  has  been  reduced  and  a 
fine  structure  appears.  When  a  Fourier-type  signal  was  used  rather  than 
a  harmonic  structure  results  again  indicate  the  uniform  spaced  property 
and  repeated  sampling  are  optimum  provided  the  signal  components  are 
Chebyshev.  If  the  observations  are  obtained  in  colored  noise,  with 
a  known  exponential  autocorrelation  function,  a  bleaching  filter 
applied  to  the  data  gives  the  solution  of  equally  spaced  samples  located 
at  the  zero  crossings  of  the  derivative  of  the  phase  shifted  highest 
order  cosine  wave. 
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APPENDIX  A 

MINIMUM  TRACE  APPROACH 


(1)  MATHEMATICAL  FRAMEWORK 

To  find  the  optimum  times  by  computer  techniques  we  will  first 
convert  the  problem  to  a  matrix  formulation  and  the  signal,  whose  spectral 
content  is  required,  will  be  modeled  as  follows  for  Fourier  and  Fourier- 
type  signals: 


XK(t) 


AR  cosWRt  +  >  BR  sinWRt 


(A-l) 


For  Fourier  signals  the  WR  are  harmonically  related,  otherwise  X^t) 
represents  a  "Fourier-type"  signal.  It  is  further  assumed  that  XR(t) 
is  contaminated  with  an  additive  process  N(t) ,  giving  the  observed 


process 


z(t)  =  XR(t)  +  N(t) 


(A-2) 


Normally,  z(t)  is  available  at  a  finite  set  of  M  times  values  {t^  :  i=l,2, 
...,  M}  where  M  is  greater  than  or  equal  to  2K+1.  It  becomes  convenient 
to  formulate  the  problem  in  matrix  form,  letting 

A  K  K 

z(ti)  =  -y  +  ^ARcosWkti  +Y^BksinWRti  +  N(ti>  i=l,2, . M 


h. *6  +  N(t . ) 
— 1  “  1 


(A-3) 


where 


[V  i  .  I 

h.  =  lx-1  cosW,  t .  cosW.t.  ...  coxW„t ,  i  sinW.t.  sinW„t.  ...  sinw,,t , 

-i  [2,  1  i  2  i  K  i,  1  i  2  i  K  i J 

and  0  =  A^^  • • •  A^  | 

The  entire  set  of  observations  can  now  be  written  in  an  M  vector  as 
follows: 

Z(M)  =  H(M)*i  +  N(m) 
where  Z (M) ,  N(M)  and  H(M)  are  given  by 

2,(m)  =  [z(tM)  Z(t^^)  •••  z(t^)] 


(A-4) 
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N(M) 

H(M) 


N(tM)  N(tM-l)  N(tl> 


,T  UT 

N  ^-1 


T 


T 


(2)  OPTIMUM  ESTIMATOR 

The  optimal  estimate  _0,  for  the  Fourier  coefficients,  will  imply 
the  best  linear  unbiased  estimate  of  9_.  Mendel  [7]  has  shown  that  for 
random  noise  N(M)  with  noise  covariance  matrix  R(M) ,  such  an  estimate 
and  its  corresponding  error  covariance  matrix  P(M),  are  given  by: 

_6(M)  =  |hT(M)  R_1(M)  H(M)|_1  HT(M)R_1(M)Z(M)  (A-5) 

P(M)  =  j HT(M)R-1(M)H(M)j -1  (A-6) 

Since  the  diagonal  terms  of  P(M)  represent  the  variance  of  the 
spectral  estimates,  a  measure  of  overall  performance,  e,  for  the  estimate 
0 (M)  can  be  defined  as  the  trace  of  the  matrix,  i.e., 


e  =  trace  [P(M)] 

Looking  at  e,  we  see  that  it  is  a  function  of  the  times  t^  selected.  What 
we  wish  to  do  is  select  the  times  t^  such  that  the  e  is  minimized. 

(3)  COMPUTER  FORMULATION 

(a)  White  noise  case.  The  performance  e  given  by  the  trace  of  P(M) 
was  found  analytically  for  the  two  cosine  case,  and  two  cosine  and  two 
sine  wave  case  for  various  number  of  samples.  The  formulas  given  in 
Appendix  B  allowed  a  direct  evaluation  of  e  that  was  minimized  on  a 
search  of  a  uniform  n-dimensional  grid. 

(b)  Colored  noise  case.  When  N(t)  is  colored  noise,  the  covariance 
matrix  R(M)  is  no  longer  diagonal.  If  the  noise  is  wide  sense  stationary 
with  autocorrelation  function  R^Ct)  given  by 

V’>  ■  *'aW 

then  R(M)  is  given  by 
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1 

-a  1 1  -t  ,1 

-a  i 

1 1  -t,  i 

e  1  n  n-1 | 

e 

In  1| 

-a 

t  ,-t. 

t  --t 

1  n-1  n| 

1 

e 

1  n-1  1 

• 

• 

• 

• 

* 

• 

• 

1  | 

• 

• 

[t.-t  i 

-a  t-t  , 

1  1  n| 

e  |  1  n-1 | 

•  •  • 

1 

A  computer  program  was  written  to  evaluate  the  P(M)  given  for  the 
special  case  that  x^(t)  contained  only  a  sine  and  cosine  wave  of  the 
same  frequency.  A  steepest  descent  algorithm  was  used  to  find  the  times 
t^,  t 2»  and  t^  that  would  give  the  minimum  trace  of  P(M). 
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APPENDIX  B 

FORMULAE  OF  THE  TRACE  FOR  PROGRAM  IMPLEMENTATION 


Two  Cosine  Waves  Case 

Let  the  signal  z(t)  be  given  by 

z(t)  =  a^cos  w^t  4  a^cot  ;^t  4  N(t) 

The  trace  of  the  error  covariance  matrix  for  the  estimates  of  and  a^ 
when  three,  four,  and  n  samples  are  taken,  can  be  shown  to  be 

(a)  three  samples  {t^.t^^t^} 

^2  2  2  2  2  2  2 

TRACE  =  —  F cos  w,  4-  cos  w- 1.  +  cos  wnt_  +  cos  w~t_  +  cos  +  cos  w^t_l 

A  L  13  12  11  23  22  2  1J 

2  2  2  2  2  2 
A  =  [(cos  w,  t_  +  cos  w,  t„  4-  cos  w, t,)  (cos  w„t_  4  cos  w„t„  +  cos  W„t,)J 
L  1  J  1  Z  11  Z  3  ZZ  Z1-* 

2 

-  (cosw^t^cosw^t^  +  cosw1t2COsw2t2  +  cosw1t1cosw2t1> 

(b)  four  samples  { t^ , t2 , , t^ } 


TRACE 


r  2  ^  ^  2  „  ^  2  ,  2 

=  —  [cos  w^t^  +  cos  w^t^  +  cos  v-fz  +  cos  wltl 


2  2  2  2  i 

+  cos  w„t,  +  cos  w„t„  +  cos  w„t„  +  cos  w2CxJ 


2  4 


2  3 


2  2 


2  2  2  2  2  2  2 
A  =  [(cos  w.^  +  cos  wic3  +  cos  wlt2  +  cos  w^t^Kcos  w2t4  +  cos  w2t3  +  COS  w2t2 

2 

+  COS  W2t^)J 

2 

-  (cosw^t^cosw2t^  +  cosw^tgCosw2t ^  4-  cosw^t2Cosw2t2+  cosw^t josw2t^) 

(c)  n  samples  t,,t~,...,t 
L  /.  n 

^2  2  2  2  2  2  2 
TRACE  =  —  [cos  w, t  4  ...  4-  cos  w, t„  4  cos  w, t,  4  cos  w„t  4  ...  4  cos  w„t„4  cos  w„t 
A'-ln  12  11  2n  22  2 

2  2  2  2  2  2  i 

A  =  rCcosVt  4. .  .4  cos  w.t,  4  cos  w,  t, )  (cos  W„ 1  4... 4  cos  w.t„  4  cos  W-t,)] 

u  in  iz  11  zn  zz  z  i  ^ 

2 

-  (cosw.t  cosw„t  4. . .4  COSW, t-C0SWot„  4  COSW, t, COSW-t, ) 

in  Zn  1Z  ZZ  11  Zl 


L6 


Two  Cosine  and  Two  Sine  Waves 
For  this  case  we  have 

z(t)  =  a^cosw^t  +  a2cosw2t  +  b^sinw^t  +  b2Sinw2t  +  N(t) 

The  trace  of  the  error  covariance  matrix  can  be  found  to  be  the  following 
for  the  four,  five,  and  n  sample  case. 

(a)  four  samples  {t  ,  t2>  t3>  t^} 

TRACE  =  a2[C0F(l,l)  +  COF(2,2)  +  COF(3,3)  +  COF(4,4)]/A 

A  =  VJX-Y-Z.  +  X, Y-Z.W.  +  Y,Z-W_X.  +  Z,W-X,Y. 

1234  1234  1234  1234 


-  Z, Y-X.W.  -  W,  Z-Y-X.  -  X,W-Z_Y.  -  Y,X-W-Z. 
1234  1234  1234  1234 


W1 

2  2  2 
=  A1+B1+Cl 

+  D1 

Xl  = 

A2A1 

+  b2b1  + 

C2C1 

+ 

D2D1 

W2 

A1A2 

+ 

B1B2  + 

C1C2 

+ 

D1D2 

X2  = 

A2  + 

2  2 
B2  +  C2  H 

-°22 

W3 

"  V3 

+ 

B1B3  + 

C1C3 

+ 

D1D3 

II 

ro 

X 

A2A3 

+  b2b3  + 

C2C3 

+ 

D2D3 

W4 

= 

+ 

B1B4  + 

C1C4 

+ 

D1D4 

X4  ” 

A2A4 

+  b2b4  + 

C2C4 

+ 

D2D4 

=  A_A, 

+  B.B,  +  C-C, 

+ 

D_D, 

z. 

=  A.  A, 

=  B.B,  +  C,  C, 

+  D.D 

3  1 

3  1  3  1 

3  1 

1 

4  1 

4  1  4  1 

4  , 

=  A_A. 

+  B_B„  +  C0C„ 

+ 

D„D„ 

Z„ 

=  a.a. 

+  B,B„  +  C.C„ 

+  D.D. 

3  2 

3  2  3  2 

3  2 

2 

4  2 

4  2  4  2 

4  . 

=  Al?  + 

2  2  2 
B^  +  CZ  +  D, 

Z„ 

=  A.  A. 

+  B.B-  +  C,C- 

+  D.D 

3 

3  3  3 

3 

4  3 

4  3  4  3 

4  . 

=  A_  A. 

+  B.B,  +  C.C, 

+ 

D.D, 

Z, 

=  k2  + 

2  2  2 
B,  +  C7  +  D. 

3  4 

3  4  3  4 

3  4 

4 

4 

4  4  4 

C0F(1,1)  =  X-Y-Z .  +  Y-Z-X.  +  Z„Y,X,  -  Z-Y.X.  -  Z,Y,X„  -  Z,X,Y- 

234  234  243  234  342  432 

C0F(2 ,2)  =  W, Y-Z,  +  Y, Z_W.  +  Z,Y.W,  -  Z.Y.W.  -  Z.Y.W,  -  Z.W.Y, 

134  134  143  134  341  431 

COF(3,3)  =  W,X0Z,  +  X,Z.W,  +  Z,X.W„  -  Z,X„W.  -  Z.X.W,  -  Z.W.X, 

124  124  142  124  241  421 

COF(4,4)  =  W1X2Y3  +  X1Y2W;j  +  Y^V^  -  Y^^  -  Y2X3W!  “  Y3W2X1 


Al" 

cosw^t^ 

B1 

as 

COSW3t2 

C1 

* 

COSW3t3 

D1 

S3 

COSW- t . 

1  4 

> 

NO 

II 

COSW2t3 

B2 

= 

cosw2t2 

C2 

* 

cosw2t3 

°2 

= 

cosw^t . 
2  4 

II 

< 

sinw^t^ 

B3 

- 

sinw^t2 

C3 

= 

sinw^t^j 

°3 

8 

sinw, t, 
1  4 

1 

< 

sinw2t^ 

B4 

- 

slnw2t2 

C4 

m 

sinw2t3 

°4 

- 

sinw2t4 

(b)  five  samples  {t^,  t 2>  t3>  t^,  t,.} 

TRACE  =  a2[C0F(l,l)  +  COF(2,2)  +  COF(3,3)  +  COF(4,4)]/A 


A  =  W.  X-Y-Z,  +  X, Y-Z-W,  +  Y-Z-W.X,  +  Z.,W-X„Y, 

1234  1234  1234  1234 

-  Z.Y-X-W,  -  W. Z-Y-X,  -  X,W0Z0Y.  -  Y,X-W-Z( 
1234  1234  1234  1234 

C0F(1,1)  =  X-Y-Z,  +  Y-Z-X,  +  Z-Y, X_  -  Z„Y,X,  -  Z„Y,X„  -  Z,Y0X0 

234  234  243  234  342  432 

COF(2,2)  =  WjY^  +  YjZ^  +  Z^W.^  -  Z^Y^  -  Z^W.  -  Z4W  Y 

COF(3,3)  =  W,X„Z,  +  X..Z-W,  +  Z,X.W„  -  Z^-W,  -  Z-X.W,  -  Z,W„X, 

124  124  142  124  241  421 

C0F(4 ,4)  =  WxX2Y3  +  X1Y2W3  +  Y  XjWj  -  Y^V^  -  YjX^  -  YjW^ 


W1 

II 

> 

M  N> 

+ 

2  2  2  2 

B^  +  +  D^  +  E3 

II 

rH 

X 

A1A2 

+  B1B2  +  Clc2  +  dld2 

W2 

=  A1A2 

+  b1b2  +  c3c2  +  dxd2 

+ 

E1E2 

X2  = 

A2  + 

2  2  2  2 
b2  +  c2  +  d2  +  e2 

W3 

=  a^a3 

+  B1B3  +  C1C3  + 

+ 

E1E3 

X3  = 

^2^3 

+  b2b3  +  c2c3  +  d2d3 

W4 

"  A1A4 

+  B..B,  +  C.C,  +  D.D, 
14  14  14 

+ 

E1E4 

X4  = 

A2A4 

+  b2b4  +  c2c4  +  d2d4 

+ 

+ 

+ 


E1E2 


E2E3 


E2E4 


Y 

Y 

Y 

Y 


1 

2 

3 

4 


A,  A. 

+ 

B,  B_ 

+ 

c,c„  + 

D,D„ 

+ 

E,E„ 

1  3 

1  3 

1  3 

1  3 

1  3 

A-A_ 

+ 

B-B- 

+ 

c0c„  + 

D-D- 

+ 

E_E_ 

2  3 

2  3 

2  3 

2  3 

2  3 

A-  + 

B 

_  +  c 

-  D-  +  E 

3 

3 

3 

3 

e 

A„A, 

+ 

B-B, 

+ 

C„C,  + 

D.D, 

+ 

E_E, 

3  4 

3  4 

3  4 

3  4 

3  4 

A  A.  +  B,B,  +  C.C,  +  D.D.  +  E,E, 

14  14  14  14  14 

^2^4  +  ®2®4  +  E2E4  D2°4  E2E4 

a3a4  +  b3b4  +  c3c4  +  d3d4  +  e3e4 

2  2  2  2  2 

A,  +  B,  +  C,  +  D7  +  E, 

4  4  4  4  4 


A^  =  cosw^t  ^ 
A2  =  cosw2t3 
A3  =  sinw^t^ 


B3  =  cosw1t2 
B2  =  cosw2t2 
B3  =  sinw1t2 
B4  =  sinw2t2 


C3  «  cosw1t3 

C2  =  C0SV3 

C3  =  sinw^t3 

C,  =  sinw-t- 
4  2  3 


D,  =  cosw.t, 
1  14 

D2  *  coxw2t4 
D3  =  sinWlt4 
D4  -  sinw2t4 


E^  =  cosw^t^ 
E2  *  cosw2t5 
E3  =  sinw1t5 
E4  *  sinw2t5 
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A.  *  sinw0t- 
4  2  1 


(c)  n  samples  n  >  5 

Same  formulas  as  (b)  except 


W1 

II 

> 

H*  NJ 

+ 

BZ  +.  . 

.+ 

X1  =  A1A2 

+  B1B2 

+  • .  .+ 

W2 

"  AlA2 

+  B1B2 

+  • .  .+ 

nln2 

X2  =  A2  + 

BZ  +. . 

,  2 
.+  n2 

W3 

“  V3 

+  B1B3 

+.  ..+ 

nln3 

X3  =  A2A3 

+  B2B3 

+.  .  .+ 

n2n‘ 

W4 

=  A1A4 

+  B1B4 

+.  . .+ 

nln4 

X4  "  A2A4 

+  B2B4 

+.  . .+ 

run 
l  i 

Yi 

=  axa3 

+  B1B3 

+.  .  .+ 

nln3 

Z1  =  A1A4 

+  B1B4 

+. .  .+ 

nln4 

Y2 

=  V3 

+  B2B3 

+.  .  .+ 

n2n3 

Z2  =  A2A4 

+  B2B4 

+. .  .+ 

n2n4 

Y3 

=  A3  + 

bz  +.  . 

.+  oZ 

Z3  =  A3A4 

+  B3B4 

+. .  .+ 

n3n4 

Y4 

=  A3A4 

+  B3B4 

+.  . .+ 

n3n4 

Z4  ’  A4  + 

b 2 

4 

.+  nZ 

0, 

= 

cosw. t 

1 

1  0 

o„ 

s 

cosw„t 

2 

2  n 

n-i 

s 

slow, t 

3 

1  0 

n. 

= 

siow„t 

4 

2  0 
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